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(57) Abstract 

A data warehouse system and method. The data warehouse includes a memory and a processor. The memory includes a database 
having a plurality of data entries. The processor includes a cache and an override engine, wherein the cache includes a subset of the 
plurality of data entries and wherein the override engine extracts data from the cache for viewing by a user, modifies the data in response 
to one or more user commands and saves the user commands to a file for later application to the database. 
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SYSTEM AND METHOD FOR UPDATING A MULTI-DIMENSIONAL DATABASE 

Rackgrounrt of the Invention 

Field of the Invention 

The present invention relates generally to database management, and more 
particularly to a system and method for manipulating facts in multi-dimensional databases. 

Background Information 

Business decision-makers operating in today's rapidly changing business environment 
need answers to a host of questions that directly impact their ability to compete in the 
marketplace. To manage and use information competitively, many companies are 
establishing decision support systems built around a data warehouse. A data warehouse 
stores a company's operational and historical data in an integrated relational database for 
decision support applications, business data access and reporting. Decision support systems 
access such databases to analyze and summarize corporate performance. 

Data warehouses employ relational database management systems that use a language 
such as SQL to retrieve rows and columns of numeric data. The systems may also permit 
access to textual files such as documents. Data may be accessed directly via user-generated 
SQL commands, or indirectly, via an interface which generates the desired SQL commands. 
Applications such as Business Objects from Business Objects S.A., France, 
(http^/ww.l>usihess6 FoieSt'WS Trees from Platinum Technology Inc., 

Oakbrook Terrace, IL, (http:^ aind Pilot's Lightship from Pilot Software 

like., Cambridge, MA^ (hftp://www:^ topical of off-the-shelf applications 

which usje brbwse Windows under th^c^ntrol :of end-users to generate the SQL code needed 
to analyze the data in the data warelipuse. o • J 

This approach, however, runs into significant performance problems associated with 
PC arid network limitations. Queries generated by inexperienced users can dominate and 
crash the database, or cause excessive network congestion. In addition, there is no 
mechanism for shifting large processes so they execute during off-peak hours. 

OLAP (OnLine Analytic Processing) technology, also called multidimensional 
analysis can also be used to access the data warehouse relatipiial database. Multidimensional 
analysis systems have been available for over 15 years, first on mainframes and then on 
client/servers. Under multidimensional analysis, data is divided into the dimensions and facts 
needed to manage the business. Dimensions for marketing applications may include 
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products, markets, distribution channels and time periods. The dimensions are used to 
reference specific points in a database. What that point represents is called a fact. As 
examples, units sold, revenue, and price are all facts. Dimensions are further described by 
attributes, such as size, flavor, location or fiscal year. Attributes also describe hierarchies 

5 within a dimension, even overlapping and inconsistent hierarchies. These hierarchies 

determine the vertical relationships within a dimension For example, in a Period dimension, 
a standard hierarchy is year -> quarter r> month -> week -> day. By defining these 
hierarchies, it becomes possible for ^plicationsHo automatically shift their 'view* up 

or down a hierarchy. This is commonly referred to ^ ^*drillin^' withm this application space, 

10 an example of this would bp shifting an annual .rpp$^'s|pt^ 

individual quarter's numbers. , ,, _ . . t • - , „ 

Multidimensional analysis allows users to, splepj, summarize, calculate, format and 
report by dimensions and.hy attobutra within j^mensions. f It c^ be used to support virtually 
any time r serie$ d^isira^ analysis, forecasting and 

15 budgeting. . ^ , , , , , .. .* ^ ^ h . ^ ^ . 

To be useful, decision support systems must support analysis based not pnly on 
historical data but also on projections for futurp activities.^ For instance, marketing may 
project sales for the next three months.,; Th$s^ fi^WpS may, then be introduced into a model 
used, to time manufacturing output over that p^^ £ Q £ f s tim^ Tp date, such , analysis has been 

20 performed usi^ig multi-dimensional datab^^, ^a^g ? ^^ Ipcjatiqns, What is n^ded is a 

system and method of extracting andmpdifymg^ an existing database which 

can be applied to a relational database in order to free r ttjg pj^zadtttionlipm^ ]the space 
limitations of multidimensional databases. In additio^ wh^t is needed is a system and 
method capable of creating reports not only based on existing information but also on 

25 projections of future activities. . > y . . lX . . _ r , . . 

SIiiminflTy of the Invention 
The present invention is a data warehouse system and method. The data warehouse 
includes a memory and a processor. The memory includes a database having a plurality of 
data entries. The processor includes a cache and an override engine, wherein the cache 
30 includes a subset of the plurality of data entries and wherein the override engine extracts data 
from the cache for viewing by a user, modifies the data in response to one or more user 
commands and saves the user commands to a file for later application to the database. 
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According to another aspect of the present invention; a data Warehouse is described. 
The data warehouse includes a plurality of workstations connected to a memory by a server . 
The memory includes a database having a plurality of data entries. The server includes a 
cache and an override engine, wherein the server operates in response to user commands to 

5 store a subset of the plurality of data entries in the cache, wherein the override engine extracts 
data from uie cache and sends the data exltracted from the cache to the client workstation for 
viewing by a user, modifies the data in response to one or more o f the user commands, saves 
the user commands to a file and ^ op^tes m resporise to 5 a commit command to modifies the 
database based on the user cbrnmahds s^orMiri me file.' ^ ! 

10 Accordmg^to yef ahothCT aspect bMe 'present invention, a method of reporting data 

from a data warehouse is described in which the steps are providirig a server and a memory 
device, storing a'datab'aie iri ^ includes a plurality of 

data erirries, exnactmg a sUte^^ 

ea!t^'m^«^r t mii&^iig me : dkta ; a^es''k(^ ; ^ : ^' - servaf in response to user 

1 5 commands, reading data from the modified data entries stored on the server and displaying 
the data to the user. . - • • 

Accord^gto yet anomer aspe^V 
based on d^ta m a data ware^ ^ providing a server and a 

memory device^ storing a database in ^memory device,' wherein the database includes a ! 

20 plurality of clata entries, exulting i'suMt bf idaH entries from the database, storing the 

subset of data i entries on the 'se^^W^^^^^^'i^^^^ on ^ server in response 
to user commaiiM 'iitii^l^-^y^iaa^a^ reading data frbm the modified data entries 
stored on the server, msplayirig the' Mala ^ tb' the user and'modifyihg the database based on the 
stored user commands. "'" " : «' " > *''*" '"' "" v '• ' ' " " *' '" A ' x ~ x 

25 According to yet another aspect of the present invention; a method of increasing the 

speed in which changes to a relational database are reflected back to the user is described in 
which the steps are extracting a subset of data from the database, wherein the step of 
extracting includes the steps of displaying a representation' of the subset of data to the user 
arid storing the subset of data in a cache, receiving a data modification command, storing the 

30 data modification command in a file and applying the data modification command against the 
subset of data stored in the cache, wherein the step of applying the data modification 
command includes the step of modifying the subset of data to reflect application of the data 
modification command. < 
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ttrief Descr iption of the Drawings 
In the drawings, y/here like numerals refer to like components throughout the several 

views, -. ... . k , * . Vf; . ( .. : > ? :i . ..... ;r r . 

Figure 1 shows a data warehouse decision support system according to .the present 
5 invention; , . , ( . * . - * 

Figure 2 shows a more detailed implementation pf the data warehouse decision 
support system of Figure 1; _ r ; , ; ; - , ^ 

Figure 3 illustrates a star sch^a .^pplem^oj^ion of a data warehouse according to the 

present invmtion;^ ; t - t , y: ,^ tq /-^ :r -; ri: r; . : ^^on^i si-O.' wy- - ; ■ 

10 Figure 4 is a more detailed desmption,o£a data warehoxisp f according to the present 

invention;... , ■ .,.,..*,;.■*. . a- 

Figure 5 shows ?w sdtpmate, embodiment pf dat^ warehouse and decision support 
system according to the pre^t.my^tipii;, and, : t r . . , r< ^ ^ 

Figures 6a ^and 6b illustrate distribution of adjusted data across levels in an unlocked 
15 and a locked system* respectively. . ,i r r; n - - Hi ; ; - . ^ 

r : , , , ; negation of ^ , ^ - 

In the following detailed ^despri^tipn o^e f pjq^^^ ^todirn^nts, reference is made 
to the accompanying drawings which form $$9^5^^ j^d in which i$ sJ^own by >vay of 
illustration specific embodiments in which the in.yer$p^ to.be 
20 understood that other embodiments may be utilized spjd stra^t^^l changes may be made 
without departing from the scope of the present i^yepl^qn^ f . ; , 

. Figure 1 illustrates a ■.cp:a»p.uter l s^ 
modify data stored in* a database. Computer system 10 ( includes a memory 12 .connected to a 
processor 14. Processor 14 includes a cache 16, a reporting tool 19 and an override engine 
25 1 8. Memory 12 is used to store a database 20. Database ,20 includes a plurality of data 
entries 26. 

In one embodiment, processor 14 also includes an instruction application process 34 
for permanently applying the modifications made within override engine 18 to data entries 26 
stored in database 20. In one such embodiment, database 20 is a data warehouse and override 
30 engine 1 8 is implemented as a multidimensional data entry/edit software engine process 
which supports the creation and adjustment of data points in the data warehouse. 
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In one embodiment, as is shown in Figure 2, cache 1 6 is implemented as a persistent 
cache which stores a subset of the plurality of data entries 26. In one such embodiment, 
override engine 1 8 extracts (data from the persistent cache for viewing by d uker, modifies the 
data in response to one or more user commands and saves the user commands to an 

5 iiistnicti6n'fiie'36"for later application to' databaise 20/ 

A user sends commands to processor 14 over command interface 22 and receives 
reports based on data extracted from database 20 tl^buj^ re^rt interface 24. In one 
embodiment, as is shown in Figure 2, command interface 22 and report interface 24 are 
implemented as a single graphicaluser ^ 

10 User action within system 10 is basically an adjustment process of data representing 

future periods in time, in one ^nb^ will have? been pregerierated by other 

systems and stored in the database 20. An example of this would be a statistically created 
forecast ^fiitiire Wluirie^ A user cam tfien "bverride" or ^ti&tiife values (i.&, facts) that 
they consider to be inaccurate. The combination* of thfe^djiisfai^ >ith the OLAP 

15 ability & present the data m virtukily any level of gr^miarity allows business professionals to 
review data within a familiar business context and use their knowledge of the business to 
refine the data warehouse values. 

As with a standard rti*a only } &^^^^^i^^si^ 9 Sys^ 10 allows end users and 
aMmistr&ors to^efine repibifts to otgi^iMe^d' pre^ntdata: "Thesfe reports are created by 

20 selecting the d^^ (tiiA^oiial ifetinl^^ks^well as the facts that contain the data needed 
to support tte decisidh or plai^ng jM^ss' 0 As ^ example, a report may contain facts such 

as c A^^ptt^ 

Region/ 'Com Synip,* and 'May 98^^ 

' : All the data 'the ii^er seeS : ori ^ testis stored' iir'a wofk* file -that wajs created for that 
25 adjustment cj^cle. The prbcess admimst^Cr work file for eadi adjustment cycle. 

For example, iii a monthly planning cycle there would be one work fife for the May 1996 
cycle, one for the June 1996 cycle, and so on. 

When the user adjusts a fact value, the information is saved in a report file, not the 
work file. In the embodiment, when the final adjustment of the fact is completed, the process 
30 administrator updates the database with the new forecast data. 

As an example, the following chart summarizes a typical adjustment cycle. In this 
case, a cycle used to create a consensus tactical forecast is shown. The chart identifies the 
required input, the steps, and the results. 
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Override Process: Tactical Forecast Cycle 



Input to the Cycle 



Steps of the Cycle 



Results of the Cycle 



Statistical forecast data, 
tactical work file, 
tactical fact. 



1. Users create reports using 
filters and dimensions. 
They work only with the 
assigned products/ 
markets/periods. t; . 

2. Users adjust last mpntjh'fe 
> tactical fact to.createtjt^s 
i: month's tactical 

They save the forecast v o ^ v ^ 
aimounts, and distribute i f . ^ 
tiie forecasts for review. , 

3. liJsere make final cb^ges % 
; sl 11 ^ save, the tactical; 

.-forecasts*- :Vi , ;: 



PreUminary reports. 



Tactical forecast 
reports for others to 
review, work with, and 
change. 



Final version of tactical 
forecasts. 



Report files, tactical 
work file. 



4. Forecasting administrator 
completes the >qycje T ; ^ " 



Management work file, 
management fact. 



As an additional example in which the: override fwction is used in a forecasting 
application, the following chart summarizes a subjective management forecasting cycle. It 
10 identifies the required inp;ut,-to^ , ; . 
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Override Process: Management Forecast Cycle 


Input to the Cycle 


Steps of the Cycle 


Results of the Cycle 


Management work file, 


1 . Users create reports using 


Preliminary reports. 


management fact. 


filters and dimensions. 






They work only with the ^ 






assigned products/ < 






markets/periods. 7 ;t 






2. Users adjust last montn s * 


Reports for others to 




management fact to 4 


review, work with, and 




Jl ' create tliis iridiith^' 5 : - nP 


change. 




management forecast. > 


i 




They sav^fefteast' 




; 


amounts^ and distribute 1 " 


1 




the forecast for review. ' 1 






3 : Users make final changes " 


Final version of 

j 




sbd save the management 1 ■■ : 


management forecasts. 




' forecast. " 11 '" " " 




Report files, ■ 


4: Forecastinglcimim^ • 


Final business fact. - 


management work file. 


completes the cycle. 





At th6 erid of rach cyble- the p data for the next 

cycle. This mvblVes saViiig the da& to t&e ^abase;p^aririg the work' file Tor the next 
cycle, and setting up the single adjustable? fact for that cycled The adjustable fact is the fact 
that can be changed or overridden. 

For example, at the end of the tactical forecast cycle, the Forecasting administrator 
updates the database with the consensus tactical forecast and sets up the next planning cycle, 
in this case a consensus management forecast. The following chart lists the steps. 
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Input to the Cycle 


Steps taken by the Forecasting 
administrator 


Results of the Cycle 


Woik file for the cycle , 
(tactical work file, and 
management work file) 


1. Appends all report files 
from all users. « t 

O Pnmmitc tVie data to the 

; £m+ M H* Lo lily UQIO %\J VLXW 

a Forecasting database. 7 
3. Sets the fact to be 
, , adjusted in the next 
. forecasting cycle, .. . . 


Integration of forecast 

rules. ; ; 

Undated Forecasting 

database. 

A new adjustable fact 
for the next cycle. 



A forecast override application such as system It) ^allows users to apply qualitative 
methods (i.e., business knowledge) to the forecasting process. Users can change values 
generated by purely statistical methods to reflect changing business conditions. 

In one embodiment, system 10 maintains a master activity file^ The master activity 
file consists of the full set of dimension information user information, and security 
10 information. It also contains detailed information about all of the facts thait were chosen by 
the process administrator to support the planning or adjustment process. It is stored in only 
one place. 

Individual users have their own configuration files and the appropriate fact data files. 

*■ a,'fiw b^u sifcr.: ^v: ik:^': 70 ?i5t?:,:v.j?f. 
When the override application is started, the master data file is read and the entire dimension 

1 5 class is built in memory for the user. The data structure has pointers to all of the data that the 
master structure points to. A system end user can view and modify any data that he has 
permission to see. The data that they are allowed to see can be configured on a user by user 
basis to ensure that they see data they are authorized to access and in a usable context. 

A lpck file is created while the work file is being processed by a user to ensure atomic 

20 usage of the data in the file. 



25 



Reports 

In one embodiment, reports are created from a template. The template is like a 
blueprint for the report. Each time a report is created, system 10 saves the blueprint. 

In one embodiment, report templates are shared among users. Each user can then 
create their own version of the report. For example, they can run a new report, modify the 
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report template; of ^drili" to create a dififerent version of the report. By drilling, they are able 
to review and adjust the data anew, with more discreet levels of detail. As an example, 
instead of simply adjusting the fact 'Planned Budget* up by 25% at the annual level, the end 
user may 'drill down' to the monthly level and review and adjust specific months values. 1 

5 ; All reports contain one or more columns and one or more rows. Each report can 

consist of multiple sections. If a report contains multiple sections, each section contains the 
same type of row and column information. ' J "5 

■ The section, column, and row names in a report reflect the dimensions of database 20. 
These dimensions give meaning to the values in databkkd 20: Tt is, therefore, important to 

1 0 understand how the dimensions work together. 

Every data value in database 20 is defined by one or more dimensions. To fully 

describe each value, a report must contain at least one element from each of the available 

-■ •'.,?<-/♦ - * ./V;. ' -J; vuci^ f:^.; ='- r . .' 1* cr ( ..r?*x*.\ v ' ■■ / ■ :,-;^;::f( 

dimensions. . , , 

Period dimensions are time intervals used for identifying and consolidating the data, v 
15 such as weekly, monthly, quarterly, and yearly intervals. Non-period dimensions describe 
other aspects of your data. Non-period dimensions may include, for example, Geo-Political, 

Product, and Business _Org. . _ , 

Each dimension can consist of hierarchical levels. For example, product may be a 
dimension. Brand and SKU are associated product dimension levels. The levels represent ^ 

20 differing degrees of detail and the paths used when you drill. 

hi one embodiment, database 20 is a data warehouse stored in one or more work files. 
Each work file can contain an unlimited number of data points. (That is, the work file size is 
limited only by the amount of physical storage.) In one such embodiment database 2 0 is 
represented as a multidimensional database. Override engine 18 can modify and view data 

25 points at any combination of levels within the multidimensional database and, as values are 
changed, the new values are allocated to all levels of the dimension hierarchies. This ensures 
that summary levels of the data still total correctly. User instructions to the override engine 
18 are not applied to the data points. Instead the instructions are stored in instruction file 30. 
Since a single instruction may change tens of thousands of rows, just storing the instructions 

30 is much faster and less expensive than storing all the changed values. Once editing is 

complete the edits can be applied to the work file by executing instruction application process 
34. In this way, large scale data warehouse updates can be performed offline while the user is 
working on other tasks. 
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By storing the instructions rather than automatically applying them, cornputer system 
10 can present the user with the results of the desired changes more quickly than in previous 
data warehouse analysis products. In addition, by using one or more work files rather than a 
predefined multidimensional database (MDDB), product, system 1Q can handle data 
warehouses which are larger in size than current database products. . 

v In one embodiment, override engine 1 8 extracts the subset of data entries 26 from 
database 20 and stores the extracted data into caphe 1 6. In one such embodiment, override 
engine 18 employs the same mechanism used _by ^ i^p^rdn.g tool 19 to extract data from 
database 20. For instance, intone such fmtw^to^ 1 8 and reporting 

tool 19 determine the appropriateJevels of,date to ^dr^t from database 20 by querying an 
OLAP object running 

reporting tool:19 1^ 36 to identify Tact tables, 

determine the levels ;they need in each fact tabtei on4 to extract the da^ from each inquired 
fact table. Override engine 1 8 then takes the report generated and pushes it into cache 16. 
From that point piv : override engine 18 receives a command,, executes the command, and 
presents the data to the user. In addition^ override engjne 1 8 . stores the command in 
instruction file 30 so that if one wants to i^runthat^ report later,; all that has to happen is that 
the commands are rej^phed to t^^ , . 

As noted above, ovemde.^ 
quickly adjusting data stored in the d^ta^w^ 

session. In one embodiment, data stored in: the^ata warehouse can be in a^y pf three states 
during such an adjustment cycle. Data that came put of database 20 is "*untouch€54/* 
Commands to alter the data are stored to instruction file 34 and are used to change the data 
being displayed to the aser. The commands do not* hpweyer, change the data in either cache 
16 or database 20. : . 

Data that's been stored back to the data warehouse (i.e., stored in database 20) is 
"adjusted** or ''written" data. Data in database 20 is modified using the instructions stored in 
instruction file 34 via a "commit" command. In one such embodiment, data stored in 
persistent cache 1 6 is invalidated during execution of the "commit" command. 

Data which has been adjusted in cache 16 but which has not yet been written to 
database 20 is in that more nebulous state in the middle where it is adjusted but not 
committed. And that's the situation where cache 16 does not match what's in the data 
warehouse anymore. In one such embodiment, the data gets to that state by executing a 
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"release"' command in engine 18. The "release" command causes override engine 1 8 to apply 
the commands not only to the data presented to the screen but also to the data within cache 16 
itself. In one sluch embodiment, commands mtist be '^released" before the resulting changes 
are "committed" to database 20. Once k set of commands are ''released", however, the 
original data is gone. 

Ifi another embodiment, overrideengirie 18 maintains a stack of old states arid the user 
can scroll back through a list of releases to recover &n older state of cache 16. 

In orie embodiment, system 10 only extracts xiata from database 20 at the start of the 
forecasting process. In siich an embodilfienti 1 s^sterii 10 only extracts data from database 20 
at the start bf tKe fbr^lfelirig ptfdc^ess. WsuclfM' anbodimbnt' Systedi'l 0 applies a three-step 
proems: w6& ftteg^ " 
generatioii acfcdmplishfed using the*e*ba<#^ 

data from da^^e 20 amd stbra (cache J I6) in the UkEX fitesy&emP 

of processor 1* ^ ! - ^A'.r^'O -v. 

llie r^mzdning phakeis (override, release, and cbmmit) use the data in the Work file to 
generate adjusted fact data. ' System fO allbwW the user to iiiteractively change individual 
proiiubt oir market values to reflect*cl^^ife ; b^iness eoridiiibhs: Eich adjustment is? r 
allocated up and down the dimension Me^bhy*(a£gregati'on levels); Allocations are v 
performed by using ^"diirieiisibri^ diiii : lS&r^Ky to identify lower level components of the 
data point l^usted^ given the new value that - 

maintains thbir relative coritnbittibirlo'-^j Users may also lofck or unlock 

individual values so that the Values do hot chaiige durinjg tne adjustnient process (directly or 
indirectly)/"' 0 * " ' * s 1 ; ' l * * r,! -' - : ■ ■••' iS ■■ * : 

Once foe 1 adjuster has cbmplete& all '^ofk for an bveitide • the adjustment is checked- 
in to the work file. The check-in process ensures that two adjusters do not adjust the same 
values and, in one embodiment, it allows a: process administrator to review the work before 
cbmrriit. ■ 1 <Pt '- ? ' x ' : "' 

The final phase of the override process is the database storage phase, or commit. 
Once the user or administrator is satisfied with the generated overrides, the overrides must be 
committed to the database. This is accomplished in the storage phase. 

As noted above, extraction is the first phase of the override process. It is the 
generation of the work file. Extraction builds a snapshot of database 20 and stores the 
snapshot in the Work File in the UNIX file system of processor 14. Extraction is generally 
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done once because it is a time-consuming process and you generally do not want or need a 
user to see that happen. In one embodiment, the extraction process is performed by an agent 
executing within reporting tool 19. The agent will typically be set up to perform the 
extraction process after database 20 is refreshed 

5 In one embodiment, the .extraction agent operates in response to user commands to 

extract the correct level of data from database 20. In one such embodiment one could set up 
system 10 to respond to a command such as **I want these markets by these products for this 
period on the horizon, by channel" (or whatever your otiier dimensions are). Another 
command syntax might be "Give t me all my pjwJu$s i^^giye mp all my markets at the 

1 0 region level oyer a selected forest hqr^^ v . r 

^ &i one such embpdirtvmVx^^h of the pera^nt -cache ,^tri^gei^.by ; th<e : aurent flag 
moving forward: one mon^ switches from January to 

February, a subset Of data is read from database 2(>^nd v a combination of facts from database 
20 and calculated facts foim^ 16. 

1 5 Any data resident in persistent cache 16 at the time of^tlie refresh that is imsaved or 
uncommitted is simply overwritt^i. , ;; f:/ ^v^^, i r ^ , 

* To build the Work File, the system, 10 must detenjiine which dimensions are (billable 
and at which aggregation levels the non-^hrillstole dime^joi^ \vdll be forecast, hi one 
embodiment,! this information is storedin the;^ 

20 For drillable dimensions; data is st9^3^.ti^^9^Eile a^.t]be:lQ^^4ey^l;4efiBed in 

the drill hierarchy, for that dimension. Fra-nqn-^lla^g dim^^Qi^^dat^ris stored only at the 
level of aggregation -specified--*!* Jh&Met^ata for j^td^^ipQ- : ; ?.v„ i r 

The Override phase allows the user t^MpB^^jffilyi^jn^^e b^se statistical forecast 
data. Data is read from the Work File g^erated >y^he extraction phase. Tjh^e user is allowed 

25 to adjust values, lock and unlock values, drill'up*sq^.4oifm.tl^,drillable dimension's 
hierarchy, as well as many other functions. ~ r 

Once an adjuster has completed a. set of override changes, these changes are released 
to the work file so that they are available for final commit to the warehouse. The release 
phase loads the base data from the work file, loads an instance file which contains the saved 

30 set of changes, and writes the new values back to the work file. 

The commit phase performs the commit for the Override process. : First, it reads the 
Work File. Next, the data values of the lowest-level decomposed data are read from the 
warehouse. These values are used to properly proportion the data as it is decomposed to the 
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lowest leveW each Dimension's hierarchy. Finally, the adjusted and decomposed data is 

written to the database. ; 

For example^ suppose the data in the Work File is stored at the regional (REG) level 
of the Market dimensioO- Further suppose that the lowest level in the Market hierarchy is 
5 market {MKT): 1 Store wiljlread the REG level dattf frohi the work file, decompose the REG 
leveldata' to me MKT leVe 1 ^ and finally write the MKT level data to the database. 

. \ , 

Th* Database Structure S. "' " 

Iii one embodiment da^ase 20 is implemented as a ndr^nbrmalized star schema. A 
simplified versidri^aslar^hem^ In a star schema, facts are 

10 stored ai data mf^^ 

mdividuar^ m dimension 

tables: In* me eii^oairnent shown ija Ffg/4/DimTable 40 is i dimension table while the 
tables iabe^ 

"Warehouse Fact" ar^ iictfabies^ 

15 Non-normalized star schema^ are designed for very fast data aggregation and 

calculation: Such speed ca^^Very^^^^ iri building the subset of data to be stored 
in cache T6f Such systems ctoi^ dowri considerably when required toperform 
incremental updaiesfeg^ asMs me^^ That is where the use 

dfcache 

20 1 ' Ttedbwiyideb^ 

replication of data across tables^ Afu^sahfe time* hdwever,' Ms^licationiof dato.-give you 
the ability to get your keys frbmihe data'warehbuse wim very small queries =that database 20 
certainly can handle quickly; * Sb a star schema 5 approach is very weir optimised for queries 
that pull a large humber'of rows i but 'of database 20. •' <> '*'<■'• ' 

25 Standard OLAP reporting tools do not have to understand multiple levels within a 

product or market hierarchy of a multi-tiered data warehouse. This is a key difference 
between override engine 18 and a standard OLAP reporting tool. Override engine 18 must 
understand multiple levels at the same time because for the purposes of an adjustment, there 
may be interdependencies on the levels themselves for a reporting tool that essentially say, 

30 "O.K., here's data at this level;" or "Here's data at the combination of these two levels." 

They're largely independent of each other and you can concentrate on one or the other at any 
given time. Because of that, override engine 1 8 does not run directly against the warehouse. 
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Instead, override engine extracts that data, performs calculations where necessary md stores 
the modified data to cache 16. , , ; r 

In order to operate correctly, however, with database 20, override ^g^ e 18 and 
instruction application process 34 must understand the structure of database 20. In one 
embodiment, as is shown in Figure 3, override, engine 18 and instruction application process 
34 extract the structure of database 20 by reviewing the contents of fvletadata table 36. 

Such an approach permits the use Qf an unlimited and easily alterable number of 
dimensions. In its simplest form, metadata I^Iq 36 includes a p/*riod dimension table 40 and 
a fact table 46. In one embodiment, eaob incudes a unique key, unique 

description; field, a level colu^ and a addition, p^^d&nension 
table includes a unique sequence number colupp, g .sgqu^iig e within ; y©ar columa and a 
currentperiod oolumn. Fact toble f 46 incudes fce#s ^i^^re identical m type^iKl structure > 

to the keys listed in dimension table 40, .... ,„ , : r , . ^ 

For example, the metadata table can be used to driv*e a drill h^rarchy that tells 
database 20 that days make up weeks, and weeks make up ^npnth^and month? make up 
quarters, quarters make up halves, halves jx^o^y^m^^^ fgur q^a^ters make up a 
year, so that the user can jump and skip things.., S^Qft^ngMhe product hierarchy. 
Override engine 18 and reporting engine 19 understand the Metadata structure and use the 
data stored in the dimension tables to extract data from database.?0. ; „ eJ .. 

; c For example, a CCTt^n table m^ maybe 
it is category, manufacturer, brand, Apd tjieo pye? i& ^Q$F? j^le are liste of SKJJs by store. 

, Another approach is £ ip;have a ^^^^^tta^.^.e^l) jno^'^^.That way 
when the next month's data arrive, it gets placed into %J*?yt fac^tal^le su^d stored in database 
20. An advantage of such an approach is that if one of the, fact tpbles .gets lost, or corrupted, 
you can reload that month. In addition, as database 20 get? bigger and bigger, its value to the 
company increases and so. does the cost of maintaining the database. Relational databases can 
scale into much larger data sizes and are much more maintainable than a corresponding multi- 
dimensional database. • , 

It should be understood that database 20 may be distributed across a number of 
computers. In one embodiment, such as is shown in Fig. 5, computer system 50 includes a 
server 52 connected to a processor 58 and a processor 62 by a network 56 ? JLn addition, server 
52 is connected to a plurality 1 through N of workstations 62. Processor 58 and processor 62 
store portions of database 20 in memories 60 and 64, respectively. 
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Iri one ^bodiment, as is shown in Fig. 5, server 52 maintains a separate instantiation 
54 of override engine 1 8, reporting tool 1 9 and cache 1 6 for each user performing the 
forecasting function. In another embodiment (not shown), a single cache 16 is used across all 
users. In either case, changes to cache 16 are maintained in a separate instruction file 30 for 
each instantiation 52 and are bnly applied to the'ciclie bn receipt of a "release" command. 

In one embodiment, processor 58 ii a multiples processor machine such as the Hewlett 
Packard HP9000 runiung an Oracle database application \Vlnle processor 62 is a Tandem 
Himalaya 128^ In one siich embddimraW ^ 

the HP inachirib while the loWesi W in the Himalaya mafchine. The Metadata 

tells the program^the appropriate pr^ces^bFto For 
example^ if data & 

deltermine'thi'^ be- liking it ! tiib Mitadati y I£ brithe other hand,* tiife user isstbring : aggregate 
data in cache 16, it will determine from the Metadata tablfe that it should extract such data 
from the Cbraele database on processor 53." ^ : li " " ^ 

In another embodiment, bveitidej e&girie 1 8 and reporting tool 19 rtm as a relatively 
thin windows clibht that essentially ji^t prdvides GUT 32. ; All the analytics^ all the storage, 
and all the real prbcessm^ : ; n , ^ 

Locking Va iner in the Database * D >>^^' o. • ^ - ^ - 

' , r iri 6n^ €&ii>6dm ability to lock values in database 20. 

As rioted dwve/bii a rfeieaise,^ie cf^Mge5 it bne lev^I of the hierarchy afre pushed down to 
each of the sub'lev"els f of the iu&arcHy: 1 Fot example; if production is increased across the 
board by 10,bo6 umts; the 16,000 uhits are (fistiibutbd proportionally across each of the v 
entities at the lowest level 'of the hiferarchy. r li ' : : ;i 

In ceitarri sitU^tid riot make serise/ For iristance, if biie of 

three manufacturing plants is operating at capacity, it makes little sense xo distribute the 
increase proportionally to the plant operating at capacity. For such situations, ovdrride engine 
1 8 includes a locking mechanism which can be used to lock the output of any of the 
manufacturing plants to a certain value. Any increases are then distributed proportionally 
across the remaining, unlocked, manufacturing plants. 

In one embodiriient, the adjustment report displays the values of the dimensions the 
user selects on a Template dialog screen. The only values that can be adjusted for the 
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forecast are found in the unshaded cells. Locked values are shaded red and non-adjustable 
values are shaded yellow. . . k 

The user can adjust a value at any level and the effects ripple through the rest of the 
product hierarchy. For example, if he or she adjusts a SKIJ value, the values of products at 
higher levels change to account for theJower level adjiKtment. Or, if the user adjusts a higher 
level value, the values for the lowpt leyel products change as needed. 

Likewise, when the user ; lpcks a value, product values both up and down the hierarchy 
may be affected. For example, if the top Jpy^l is. lock(^ at 10Q, all lower values must total 

The dUgram ! shown or propprtional, fitting 

based on eurre&t forecast .data. J~h$ current adju^ttjae^t is retrieved at mraufaetae level 
70 and* after m ^justo ^rand jevel 72 toJSKU level 

t If,JiGwever, thevmanufacttu-e product y^ue^wasjocked at level 70 arid one Brand 
product value was adjusted firpip; 50 to 60, the adjus^OTt jand its effects are shown in Fig. 6b. 

. As nptedj^ve, in pne embo<Km$nt both reporting iPOl, 19 and override engine 18 
have the, ability to hrnit the effect qf&rnao^fc^ from database 20 

does not include the entire hierarchy for atydipei^piiSj, Ec^ instance, pne may ^are about 
multiple levels of product dimension, b^twould ^1^:^ Jj^it rtli^ -^^Hpt ^j^J^^^ 1 ^ to a 
particular level (e.g.^ i^ake the market adjus^^ ) ;? In one s^ch ^^odiment, 

the user can: select the level at which data i$ adjusted. , ; example, pne WQyXd tell override 
engine 1 8 .to, "Make your product dimc^ic«a (tollable, flia£p .your, market dimension non- 
drillable, make your period dimejision non^llafcle " Adjustments then rnust only be driven 
down the product hierarchy for display; the override engine doesnpt have to /dry to drive it 
down the other hierarchies. As you add multiple drillal?le dimensions, the amoynt of work 
that has to happen when you make adjustments expands exponentially. Not only does it go 
down product, but it has to go down product for every market, or every submarket, or every 
submarket of every submarket and you end up with a huge matrix of numbers to keep track 

Of , . ■ • : - ~ 

Drilling allows the user to display the fact values at different product levels, both up 
and down the product hierarchy. With drilling, one can see the aggregate values of a group of 
products, or can identify the specific values that went into the aggregate amount. 
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t>rili&g up works with any product where there are other products at a higher level. 
For example, one can drill up on a value if he or she is at SKU level 74 shown in Fig. 6a. 

Finally, a user can drill down with any product if he or she is at a level above the 
lowest level, SKU. For example, one can drill dbWn if at manufacture level 70 in Fig. 6a. 

If, for instance, you want to adjust the amount of a particular soft drink sold in a 
particular city, you could either make that a drillatole thing or a iion-drillable dimension. You 
can do it in one of two ways. One is you can set your niaricet diriiension as (billable, and then 
poll the whole market hierarchy in order to drill down -to the city and make the change. If, 
however, you did not need that flexibility in drilling and the associated performance impact 
of allocating the chariges ttri-ougji tfr;St <3i^ market dimension 

*noi^lfeble^ci only ^ee r dati af A^fcity tevell 1 Then you caii jvtst grab whatever city you 
wanted and make your adji^^ 

level below city, once you do a "commit" it is going to drive the numbers to the bottom of all 
the hierarchies (In bhe ^l^dimeirtj ^ override; ragine drives the numbers down to the bottom 
of iall the hierarcttes by m^ percfeit^of contribution to the total.) 

For exainple,' if giv^ri a market Meratchy including regidris, cities within those regions 
and stores witMhtobse citibs^ you were gorng to inddiiy units at the regional level, yc>u only 
need to maintain data in cache 16 af that levfel ^Tado-this, override 'engine 18 initiates a 
transferor data ait Iflie atomi^levfel^ te$#Ss' tfie ; data and pushes it up to the regional leviel;^ r 
'the ag^eg&tecl r^onal data is ffieh^ from the user are- 

applied against that agj^eigate da^ arc re&3y to store the changed data, a 

"commit" coi^ahd is ^x^ut^^afia'the niodihcatiohs get drilled down to the atonric level in 
database 20. llie result i^ that user commaiids get texiEfcuted quickly at * e regional level and 
get stored accurately at the atomic l'evei;' v * ' J ° : ' * " 5 ; v 1 ^ - ; ; 

For e*ariipte^ one might dtf^ 
a region such as the Great Lakess tenitoiy. The change would be made at the territory level 
and, when committed, it is distributed tb the cities' as a function of the percentage they 
contribute to the regional total. ! ' * 

No matter the level where the adjustment is made, once the warehouse is updated, it is 
updated at the lowest atomic level and then in this case for reporting engine 19, then it can 
automatically pick up a report of it based on market hierarchy, period hierarchy, etc. All that 
is a trade-off between speed and flexibility. You can go ultimately flexible if you've got 
enough iron behind the thing to drive it. And if you do not, then you can cut back. 
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Also from a security perspective, quite often if it is sales people ttmt are using it for 
the adjustments, they do not want them to see other territories, for instance, other people's 
clients, other sales guys' clients. Otherwise they start playing games that you do not want 
them playing. It is a combination of those things. 

Override engine 18 understands all of importing tool 19's filters and calculated facts; 
it also understands the hierarchies of system 10. Essentially everything in the Metadata that 
can be defined, is read by reporting tool 19 and oyerride engine 13 and they will us? those 
definition. So you can create a calculated fact ^r^itfng tool 19 and override engine 18 
will automatically see it and know how, to, use it^/; T , v . , -| ; , ; . : 

(Antexample of a tabulated fast \vpul4j^ is calculated 

as units times; price. You would noVvti^^ fostead, ypu calculate it 

off of the two that you did stor§<p#c# ^ frcUs forecast error. 

Facts calculated asa function; pf w actual fact ai^ ^fQrjec^ted factf are derived, not stored. 
Override engine :\ 8 automatically rea^js the definitipn pf e aqh calculajejd fapt put of the 
Metadata and applies them on the fly to the data reajd from, diabase 20.) ... f ..... . 

In one embodiment, forecasts aip stored as separate (Jatabases within database 20. For 
instance; one could include a dimension lately warehouse. It can, 

therefore; be critical to have the.abilityto add extr? ^dim^s^ons to systems 10 and 50. 



Cache ' 




As was noted in connection with Figure 2, in one embodiment cache 16 is a persistent 
cache stored as a B-tree in a UNIX file system on processor 14. The B-tree allows you to 
essentially go with a much more compact data storage in situations where the matrix of data 
is populated sparsely (such as in, for example, customer centric databases) yet, at the same 
time, a B-tree implementation does allow you to get reasonably fast access times. 

In one such embodiment, the persistent cache is implemented using standard Rogue 
Wave B-tree code. The performance of the Rogue Wave B-tree code can be enhanced by 
replacing the standard I/O stream (which is write character by character) with an I/O stream 
which writes data in large blocks of data. 

In addition, with a B-tree implementation, one can trade off size for speed by 
increasing or decreasing the number of branches and subbranches in the tree. That is, the 
deeper you make the B-tree, the slower your access time but, at the same time, the smaller the 
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actual physical "storage. In one embodiment, the structure of the B-tree is optimized for size 
over speed. 

In more densely packed matrices of data; cache 16 canbeimplemented as a set of file- 
based arrays. Such arrays are very fast, requiring only simple calculations to get to the right 

5 locations. They can be, however, very inefficient space-wise. Typically, if one of the 

dimensions of the data warehouse is customer, arid the number of potential customers is large, 
you are better off selecting a B-tree implementation for cache 16. 

Override engine 1 8 allows the user to grab slices of data-ftom a database that may be 
too big to comprehend in its entirety. Override engine 18 allows you to slice a portion of the 

10 database bui wh%h : sometimes in 

arid inteiiigen^iy through the user of me^^ata^ The portion extracted' can be dealt with either 
within or outside me database quickly k^effici^ m P laee ' "' 

override engine 18 automatically 'and se^ The * 

result is a database tool which eliminates the query tiiaffie botiieneck of traditional approaches 

15 to relational database management systems; * ' v ' ■ • •'•'< • " '- : ••'•>•;"' 

Although specific embbdimerits tiavebeen illustrated and 'described herein, it will be 
appreciated by those of ordinary skill m the arrthat any iOTangeinent which is calculated to 
aclneve'the'same purpc^-may'Wst^^raW-foir^fee specific -•embodiment shown.v This-- :■> 
application is intended to cover any adaptations or variations of the present invention. 

20 Therefore, it is intended that this invention be limited only by the claims and the equivalents 
•mereof.'""^ ■ ^ ^ *™ n: - ^ n ••■ '■ ! : < ^ •■■ • 
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What is claimed is: ; 

1 . A data warehouse, comprising: 

memory, wherein the memory includes a database having a plurality of data entries; 
a processor connected to the memory, wherein the processor includes: 
: .• -a cache; and v ... . . r .. y-, ........ t . . .. 

an override engine; 

wherein the cache includes a subset of the plurality of data entries and wherein the 

0 

override engine extracts data from the cache for^ae^wing by a user^ modifies the data in 
response to one or mere user coprmiands.fmd say^ commands to. £ file for later 

application to the database. s f7 , : r ,- l>u 

2. A data warehouse, comprising: ^ r:i . ; j } ; : . i: . i, : .,, v ^ 
memory, wherein the.mempry includes ^ d^al^e having a plip-ality of dat^ entries; 
a server connected to the memory, wherein the processor includes: 

a cache; and • ... ; . : <y u r - t ri -.-^ - : . 

an override engine; and ^ c - v^t. r r t ; ■?=. r , > 
a client workstation connected to th^sjsry^ ; * ; V . ;r ; 1 .... i 
wherein the server operates in response to user commands to store a subset of die 
plurality of data entries in the c^che, wherein the ^ the cache 

and sends the data extracted from the cache ^toe-clie^iw^^ by a user, 

modifies the data in response to one or more qf;^f user ,coin|naiids, saves the, user commands 
to a file and operates in response to a commit cqmiriand tp piodifies the database based on the 
user commands stored jn the file. v - ha* r !«rriu*; j. ^-.v 

3. A method of reporting data from a data warehouse; the method comprising the steps 
of: 

providing a server and a memory device; ? ^ 
storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extracting a subset of data entries from the database; 
storing the subset of data entries on the server, 

modifying the data entries stored on the server in response to user commands; 
reading data from the modified data entries stored on the server; and . 
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displaying the data to the user. 

4. A method of forecasting, comprising the steps of: 
providing a server and a memory device; 

storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extracting a subset of data entries from the database; : - 
storing the' subset of data entries on the ^ iseii^er; 

modifying the data entries stored ^ oh tiie SerVdHn response to user commands; 
storing the user commands; 

reading data from the modified data entries stored on the server; 
displaying the data to the user; and :}*ri<-zuii,*>u - ^ 

tiA '- v - 'modifymgiili^ diatab^e b&ed' : 6iri tlie-storeduser commands. :t ^ \ ; ; 

5. A method of increasing the speed in which changes to a relational database are 
reflected back to the user, comprising the steps of: 

extracting a subset of data from the database] wherein the step of extracting includes 
the steps of: - ^ :/ ? ^ri./.-uv. - '-^ f 

- ~- " ^ v ,; ^- storing :v... \ , , ■ ■ ■ 

-<? a.?. r^ceiving^a ! data mbdifi^tionxSnMand^ "~ ^ : 
• storing thb data ^ 

applying the data modification command against the subset of data stored in the 
cache, wherein the step of applying the data modification command includes the step of 
modifying the subset of data to reflect application of the data modification command. 

6. The method according to claim 5, wherein the step of applying the data modification 
command further includes the step of modifying the representation of the subset of data to 
reflect application of the command. 

7. The method according to claim 6, wherein the step of applying the command further 
includes the steps of: 

waiting for a "commit" command; and 
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on receipt of the "commit** command, modifying data in the database to reflect 
application of the data modification command. 

8. The method according to claim 5, wherein the step of applying the command further 
includes the steps of: : 

waiting for a "commit*' command; and ; 

on receipt of the "commit" command, modifying data in the;database to reflect 
application of the data modification command. ^ 

\ >" J " ' ^ \ 

9. A data structure for updating a database, comprising: 

a plurality of user data-modification commands, wherein the plurality of user data- 
modification commands provide information for modifying information in a database (20). 

i 

10. The data structure of claim 8, wherein: j 

i 

the plurality of user data-modification commiands provide data-modification 
•information for application to aggregated regional dpta stored in a cache (16), and 
modification information that gets drilled down to ah atomic level in the database (20). 
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